Jumping Rivers

• Data science
  • Python/ R, machine learning, dashboards, API’s
• Data engineering
  • Data pipelines, server health and security, managed RStudio (Posit) services
• Training
  • Python, R, Git + many more

Outline

1. Common issues faced
2. Some solutions
3. How can Version Control Systems help?
4. Demo: Intro to GitHub

Tracking versions

Collaborating

• Multiple people making edits to the same file
• Everyone has their own version
• Duplicated work
• Giving/ getting feedback
• Sharing work

Backing up

How to combat these issues?

Manual Versioning

• Standardized naming convention

  • 0.1-ptg-exciting-report.txt

  • 0.2-ptg-exciting-report.txt

  • 1.0-ptg-exciting-report.txt (finalised version)

  • 1.1-ptg-exciting-report.txt (revision)

Manual versioning

Structured folder of copies

<project-name>
|   CHANGELOG.txt
└───current
│   │   code.R
|   |   report.txt
└───2023-06-13
|   |   code.R
|   |   report.txt
└───2023-06-12
|   |   code.R
|   |   report.txt

CHANGELOG.txt

* _2023_06_13
  * Fix: resolved a minor bug which prevented...
  
* _2023_06_12
  * Feature: added a new function which does...

Beyond manual versioning

• Versioning manually requires a lot of self-discipline

• Combining versions manually is hard work

• Might need a version control system such as Git

What is Git?

What are GitHub and GitLab?

• Think about e-mail… we could host our own e-mail server, but that’s usually too much effort

• Git is like e-mail

• GitLab & GitHub are like gmail or yahoo

• There are other git hosting services

• Today we’ll focus on GitHub

Setting up a GitHub account

• It’s free!
• https://github.com/

Authentication

• GitHub uses personal access tokens (PATs) as a secure method of authentication

• A PAT is an auto-generated strong passphrase (not ‘monkey123’!)

• The user can set an automatic expiry date and different permissions for each token

• This guide explains how to set one up in GitHub (make sure to select ‘repo’ when setting the scopes)

Creating a Git repository (project)

There are a couple of different ways that we can start using a Git repository:

• Clone an existing repository
• Create a new repository from scratch

Cloning a GitHub repository

• Downloads a copy of that repository to your machine

• Takes the latest snapshot available (all of the files) from GitHub

• By default saved in a folder with the same name as the repository

Cloning a GitHub repository

Workflow

1. Making changes locally
2. Updating the remote repository

Making changes locally

Making changes locally

Making changes locally

Commits

• Commit takes a snapshot of all your files

“Using a Git commit is like using anchors and other protection when climbing. If you’re crossing a dangerous rock face you want to make sure you’ve used protection to catch you if you fall. Commits play a similar role: if you make a mistake, you can’t fall past the previous commit. —”

R Packages, Hadley Wickham.

Making changes locally

We are going to:

• Edit & save some files in our working directory

• Add these changes to the staging area

• Commit the changes in the staging area to our local repository

Updating the remote

Updating the remote

Now we are going to:

• Check for changes by any colleagues by pulling
• Push our local changes to the remote repository

So to recap

So, about merging…

• Sometimes it’s not as simple as we just said

• Git can often automatically pull and merge changes

• If it can’t you get a merge conflict

  • Often occurs when two people have edited the same line of a file
  • Git then doesn’t know which commit takes precedence

• If a merge conflict occurs, we have to manually fix it.

Not just version control!

• Project management

  • Issues
  • Labels
  • Milestones

• Continuous Integration

• Automated deployments

• Security

Branching

• Here we pushed changes to the remote main branch

• main should always have working code

• In reality we would make commits to a separate branch & then create a pull request